feat: tenant-isolated BYO routing contracts for SaaS by jmlago · Pull Request #104 · genlayerlabs/unhardcoded

jmlago · 2026-09-05T13:11:22Z

Release safety — draft, do not merge or deploy yet

This PR publishes reviewable code only. It must not enable auto-merge or approve the existing PRD environment.

The production release chain was verified read-only on 2026-09-05: a push to unhardcoded/main starts notify-ci behind a manual PRD approval. Approval dispatches unhardcoded-ci, which builds both images and triggers an automatically merged infrastructure bump in devexp-argocd-apps; ArgoCD then updates the API, the single Recreate provider/control pod (AntSeed + Codex), and analytics. Production was running v-5d99074. Disabling the control-plane variables does NOT make all shared code/schema changes in this PR a no-op.

Before release: establish an independent SaaS image lane and isolated deployment. Do not reuse production wallets, provider keys, Bedrock Pod Identity, database, PVCs, or the existing public endpoint. Infra/CI PRs for that lane are not implemented by this PR. If main must remain independently releasable, establish a SaaS release branch or qualify these changes for the existing router before merging.

Scope

Includes and extends the original integration in feat: external control-plane protocol (key resolve, usage API, tenant BYO keys) #72. This PR is the replacement review candidate for this delivery; do not merge feat: external control-plane protocol (key resolve, usage API, tenant BYO keys) #72 as an additional prerequisite. The older PR has been left untouched.
Records the previously staged upstream integration as a two-parent merge (1ec91ce): upstream's 77 changed files retain their history rather than appearing as a new SaaS feature commit.
Adds tenant-isolated BYO connections, explicit per-request Bedrock credentials and tenant discovery, and a customer-owned AntSeed gateway restricted to inference and read-only discovery.
Enforces published route:* contracts and authorized cost/speed/reliability preferences, rejecting client policy/flow/timeout bypasses.
Uses the existing Lua engine for normalization, admission, ranking and execution. No engine source changes.
Adds internal catalog/connections/preview/test endpoints, trusted tenant context, and bounded routing summaries. The host store gains nullable calls.routing_summary.
Shared ingress, shim, host and provider adapters are modified; this is not an isolated UI-only change.

Known boundaries

Request-local Lua runtimes; no persistent per-tenant circuit-breaker state or customer-specific durable AntSeed peer-health history yet.
No load-test or production-readiness claim. A control-plane outage rejects new SaaS calls; application-key resolution retains the existing TTL/stale-grace behavior.
Usage accounting is best-effort, not billing-grade.
The optional buyer gateway token can spend through inference, but cannot withdraw funds or export wallet keys.

Verification

Dedicated disposable local PostgreSQL: 859 passed, 2 skipped; no paid provider calls.
Existing deprecation/OpenAPI warnings and pool cleanup warnings remain; process exit code was 0.
git diff --check clean.
CI on the draft PR must still be reviewed. Local tests do not replace built-image checks.

Dependencies and merge order

Companion PR: genlayerlabs/unhardcoded-cloud#2, branch feat/saas-byo-workspace, pinned to 82ce727.

Review the isolated release lane and validate exact images independently of the running router.
Merge this dataplane PR only when the above release implications are accepted; do not approve the legacy PRD release for this SaaS.
If squash/rebase merging changes the commit ID, repin cloud to the resulting accepted commit and rerun checks.
Merge the companion cloud PR.
Deploy only through separately reviewed SaaS infrastructure PRs. Neither this PR nor the cloud PR provisions that infrastructure.

Summary by CodeRabbit

New Features
- Added optional SaaS routing with tenant-specific provider access, credentials, policies, previews, failover, and capability filtering.
- Added external control-plane support for authentication, route resolution, tenant configuration, and usage metering.
- Added bring-your-own provider support for AWS Bedrock and Antseed buyer connections with secure HTTPS handling.
- Added an optional customer-owned gateway for snapshots and chat completion forwarding.
- Added authenticated usage totals and recent-call reporting endpoints.
Security
- Added tenant isolation, credential safeguards, request validation, and protected internal endpoints.

… BYO keys) Adds an optional "bring your own control plane" integration, entirely off unless CONTROL_PLANE_URL + CONTROL_PLANE_INTERNAL_SECRET are set: - control_plane_client.py (new leaf module): resolve_key with positive/ negative caching, single-flight, and stale-grace served only while the control plane is unreachable; tenant_env (allowlisted, cached, fail-soft to platform keys); env_get — a request-scoped credential chain (ContextVar tenant map, then process env). - Ingress: _caller_auth_async falls through to the control plane only on a pure local miss (local stores stay authoritative; explicit local status=inactive is an operator kill-switch for a tenant slug). Plan rate limits ride the resolve response into the existing _rate_ok meta. x-llm-router-tenant / x-internal-secret are stripped from client input; the tenant header is stamped only from the authenticated resolve. - internal_api.py (new): GET /internal/usage[?bucket=day] and /internal/usage/recent, gated by x-internal-secret (hidden 404 while unconfigured), backed by new host_store.usage_totals (includes cached tokens) and recent_calls(caller=...). - Router: per-tenant BYO provider credentials — the shim activates the tenant env from the trusted header; serve.py threads control_plane_client.env_get into the openai-compatible + anthropic + google adapters (bedrock/codex/antseed stay platform-only; background source pollers never see tenant keys). _rate_ok/_route_allowed now accept the already-resolved meta, dropping a duplicate consumer_keys read per proxied request.

Record the previously staged upstream integration as a two-parent merge, preserving the original control-plane commit and upstream history. SaaS working-tree changes are committed separately.

Enforce published route revisions and authorized task preferences through the existing Lua engine. Add request-scoped provider discovery and credentials, Bedrock BYO, an optional customer-owned AntSeed gateway, internal preview/test endpoints, and bounded routing summaries. Validated locally: 859 tests passed, 2 skipped. Production deployment is not authorized by this commit; shared ingress, host, adapter and schema changes require separate release review.

coderabbitai · 2026-09-05T13:11:41Z

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: 26619e26-2b8c-48eb-a615-26de78b19a1b

📥 Commits

Reviewing files that changed from the base of the PR and between 13d2bd6 and 82ce727.

📒 Files selected for processing (33)

.env.example
antseed/byo-gateway.js
antseed/byo-gateway.test.js
antseed/entrypoint.sh
auth_proxy.py
byo_http.py
compose.yml
control_plane_client.py
host_store.py
internal_api.py
llm_router_host.py
provider_adapters/aws_credentials.py
provider_adapters/bedrock.py
provider_adapters/openai_compatible.py
provider_connections.py
providers.py
route_contract.py
saas_routes.py
serve.py
shim.py
sources/antseed.py
sources/bedrock.py
tenant_providers.py
tests/fixtures/managed.lua
tests/fixtures/saas.lua
tests/test_antseed_node.py
tests/test_auth_proxy_control_plane.py
tests/test_control_plane_client.py
tests/test_host_store.py
tests/test_saas_byo.py
tests/test_saas_managed.py
tests/test_saas_routes.py
tests/test_tenant_env_shim.py

📝 Walkthrough

Walkthrough

Changes

The change adds optional external control-plane authentication, published route enforcement, internal usage metering, tenant-scoped SaaS routes, BYO provider credentials, secure buyer egress, and an authenticated Antseed gateway.

Control-plane integration

Layer / File(s)	Summary
Control-plane client and metering contracts `control_plane_client.py`, `host_store.py`, `internal_api.py`, `.env.example`, `compose.yml`, `route_contract.py`	Adds cached control-plane resolution, tenant environment activation, published route retrieval, bounded routing summaries, usage aggregation, and authenticated internal usage endpoints.
Ingress authentication and route enforcement `auth_proxy.py`, `tests/test_auth_proxy_control_plane.py`, `tests/test_control_plane_client.py`, `tests/test_host_store.py`	Adds local-first control-plane key resolution, tenant route restrictions, published contract application, protected forwarding headers, and integration coverage.

Tenant-scoped SaaS routing

Layer / File(s)	Summary
Tenant host isolation and provider catalog `llm_router_host.py`, `provider_connections.py`, `saas_routes.py`, `tests/fixtures/*`, `tests/test_saas_routes.py`, `tests/test_saas_managed.py`	Adds tenant child hosts, provider allowlists, catalog connection metadata, intent compilation, policy previews, preference handling, and bounded failover.
SaaS middleware and HTTP endpoints `saas_routes.py`, `shim.py`, `serve.py`	Installs tenant middleware and exposes catalog, connections, preview, and test endpoints with request-local host and credential context.

Tenant BYO providers

Layer / File(s)	Summary
Credential-scoped provider adapters `tenant_providers.py`, `provider_adapters/`, `providers.py`, `serve.py`, `sources/`, `byo_http.py`	Loads tenant Bedrock and Antseed connections, prepares scoped offers, requires explicit AWS credentials, and routes buyer requests through HTTPS DNS-pinned transport.
BYO isolation validation `tests/test_saas_byo.py`, `tests/test_tenant_env_shim.py`	Covers credential isolation, provider discovery, funding gates, private DNS rejection, TLS hostname preservation, streaming context, and authorization headers.

Antseed gateway

Layer / File(s)	Summary
Gateway server and wiring `antseed/byo-gateway.js`, `antseed/entrypoint.sh`	Adds authenticated snapshot access and pinned chat proxying, then starts the gateway as a separate process.
Gateway validation `antseed/byo-gateway.test.js`, `tests/test_antseed_node.py`	Tests authentication, unsupported endpoints, pin enforcement, header forwarding, and authorization-header stripping.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant AuthProxy
  participant ControlPlane
  participant SaaSRoutes
  participant ProviderAdapter

  Client->>AuthProxy: Send tenant request
  AuthProxy->>ControlPlane: Resolve key and published route
  ControlPlane-->>AuthProxy: Return tenant and route contract
  AuthProxy->>SaaSRoutes: Forward authenticated request
  SaaSRoutes->>ProviderAdapter: Execute with tenant credentials
  ProviderAdapter-->>Client: Return provider response

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/saas-byo-contracts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

jmlago added 3 commits July 4, 2026 01:10

Merge main into SaaS integration baseline

1ec91ce

Record the previously staged upstream integration as a two-parent merge, preserving the original control-plane commit and upstream history. SaaS working-tree changes are committed separately.

jmlago marked this pull request as ready for review September 5, 2026 13:56

jmlago merged commit 3e72abf into main Sep 5, 2026
3 of 4 checks passed

This was referenced Sep 5, 2026

feat: external control-plane protocol (key resolve, usage API, tenant BYO keys) #72

Closed

feat: enforce SaaS project scopes and isolate environment usage #105

Merged

Navigation Menu

Uh oh!

feat: tenant-isolated BYO routing contracts for SaaS - #104

feat: tenant-isolated BYO routing contracts for SaaS#104
jmlago merged 3 commits into
mainfrom
feat/saas-byo-contracts

jmlago commented Sep 5, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Sep 5, 2026 •

edited

Loading

Review failed

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

jmlago commented Sep 5, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Release safety — draft, do not merge or deploy yet

Scope

Known boundaries

Verification

Dependencies and merge order

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review failed

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jmlago commented Sep 5, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Sep 5, 2026 •

edited

Loading